INTERFACING VIA HEADS-UP DISPLAY USING EYE CONTACT

• US 20160246054A1
• Filed: 08/26/2015
• Published: 08/25/2016
• Est. Priority Date: 02/23/2015
• Status: Active Grant

• Current Assignee: International Business Machines Corporation
• Original Assignee: International Business Machines Corporation
• Inventors: Bedell, Stephen R. Jr., Gullo, Richard S. Jr., Myers, Andrew C., O'Meilia, Casey M.
• Granted Patent: US 9,658,689 B2
• Time in Patent Office: Days
• US Class Current: 1/1
• CPC Class Codes: G02B 2027/014 comprising information/image processing systems
• CPC Class Codes: G02B 2027/0187 slaved to motion of at least a part of the body of the user, e.g. head, eye
• CPC Class Codes: G02B 27/01 Head-up displays
• CPC Class Codes: G02B 27/0101 characterised by optical features G02B27/0172 takes precedence
• CPC Class Codes: G02B 27/017 Head mounted
• CPC Class Codes: G06F 3/013 Eye tracking input arrangements G06F3/015 takes precedence
• CPC Class Codes: G06F 3/04817 using icons graphical or visual programming using iconic symbols G06F8/34
• CPC Class Codes: G06F 3/14 Digital output to display device ; Cooperation and interconnection of the display device with other functional units
• CPC Class Codes: G06F 3/167 Audio in a user interface, e.g. using voice commands for navigating, audio feedback
• CPC Class Codes: G06V 40/193 Preprocessing; Feature extraction
• CPC Class Codes: G09G 2370/022 Centralised management of display operation, e.g. in a server instead of locally
• CPC Class Codes: G09G 5/003 Details of a display terminal, the details relating to the control arrangement of the display terminal and to the interfaces thereto specific for a CRT G09G1/165; for a flat panel G09G3/2092
• CPC Class Codes: H04L 67/01 Protocols
• CPC Class Codes: H04L 67/06 specially adapted for file transfer, e.g. file transfer protocol [FTP]
• CPC Class Codes: H04M 1/72412 using two-way short-range wireless interfaces

Abstract

According to one embodiment of the present invention, a system for device interaction via a heads-up display determines an area of sight of a first entity employing a first heads-up display, and establishes interaction between at least one first device and the first heads-up display in response to detecting at least an impact of the determined area of sight on at least one second device. Embodiments of the present invention further include a method and computer program product for interacting with a device via a heads-up display in substantially the same manners described above.

Claims

1. A method of device interaction via a heads-up display comprising:

determining an area of sight of a first entity employing a first heads-up display;

establishing interaction between at least one first device and the first heads-up display in response to detecting at least an impact of the determined area of sight on at least one second device.

2. The method of claim 1, wherein the area of sight includes one of a line of sight and a cone of sight.

3. The method of claim 1, wherein the interaction between the at least one first device and the first heads-up display includes one or more from a group of:

controlling one or more from a group of the at least one first device and the first heads-up display, viewing a status of one or more from a group of the at least one first device and the first heads-up display, viewing a message associated with one or more from a group of the at least one first device and the first heads-up display, a server-client relationship between the at least one first device and the first heads-up display, transferring a file between the at least one first device and the first heads-up display, initiating a connection for audio communication between the at least one first device and the first heads-up display, and authentication of the first entity to one or more from a group of the at least one first device and the first heads-up display.

4. The method of claim 1, wherein the first heads-up display is coupled to the at least one second device, and establishing the interaction comprises:

presenting an interface of the at least one first device on the first heads-up display;

manipulating the presented interface based on one or more from a group of the determined area of sight of the first entity on the at least one second device coupled to the first heads-up display and at least one voice command; and

controlling the at least one first device based on the manipulation of the presented interface.

5. The method of claim 1, wherein the at least one first device includes one or more from a group of a second heads-up display and a processing device.

6. The method of claim 5, wherein detecting at least an impact of the determined area of sight comprises one or more from a group of:

detecting at an electromagnetic signal detector of a corresponding one of the at least one second device associated with the first heads-up display electromagnetic signals transmitted from a transmitter associated with the at least one first device; and

detecting at an electromagnetic signal detector of a corresponding one of the at least one second device associated with the at least one first device electromagnetic signals transmitted from a transmitter associated with the first heads-up display.

7. The method of claim 6, wherein the transmitted electromagnetic signals associated with the at least one first device and the first heads-up display each include information to establish the interaction.

8. The method of claim 5, wherein the first heads-up display includes an emitter to transmit signals along a direction of sight of the first entity and each of the at least one first device is associated with a corresponding one of the at least one second device comprising a receiver to receive the signals, and detecting at least an impact of the determined area of sight comprises:

detecting at the receiver associated with the at least one first device the signals transmitted from the first heads-up display.

9. The method of claim 8, wherein the signals transmitted from the first heads-up display include information to establish the interaction.

10. The method of claim 1, wherein the at least one first device includes a second heads-up display, and detecting at least an impact of the determined area of sight comprises one or more from a group of:

detecting at a receiver of a corresponding one of the at least one second device associated with the first heads-up display signals transmitted by an emitter from the second heads-up display along a direction of sight of a second entity; and

detecting at a receiver of a corresponding one of the at least one second device associated with the second heads-up display signals transmitted by an emitter from the first heads-up display along a direction of sight of the first entity.

11. The method of claim 10, wherein the signals transmitted from the first and second heads-up displays include information to establish the interaction.

12. The method of claim 8, wherein the first heads-up display further includes an image capture device, and determining an area of sight comprises:

tracking the area of sight of the first entity via the image capture device; and

aiming the emitter to transmit the signals in the direction of the tracked area of sight.

13. The method of claim 8, wherein the first entity employs a contact lens with a plurality of objects defining the area of sight, and determining an area of sight comprises:

ascertaining the area of sight defined by the plurality of objects; and

aiming the emitter to transmit the signals in the direction of the ascertained area of sight.

14. The method of claim 1, wherein a plurality of first entities each employ a corresponding first heads-up display, and the at least one first device comprises a second heads-up display including the at least one second device, and wherein determining an area of sight further comprises:

determining an area of sight of each first entity employing a corresponding one of the first heads-up displays;

wherein establishing the interaction further comprises;

establishing interaction between the second heads-up display and each first heads-up display in response to detecting impact of the determined area of sight of a first entity of that first heads-up display on the at least one second device of the second heads-up display.

15. The method of claim 1, wherein the at least one first device includes a second heads-up display employed by a second entity, and the at least one second device includes a common device impacted by determined areas of sight of the first entity and the second entity.

16. The method of claim 1, wherein the at least one first device includes a second heads-up display employed by a second entity, and the at least one second device includes an initial device impacted by the determined area of sight of the first entity and another different device, remote from the initial device, impacted by a determined area of sight of the second entity.

17. The method of claim 1, wherein each of the at least one first device is represented as an icon in the first heads-up display to provide information in response to establishing the interaction with that first device.

18. The method of claim 1, wherein the first heads-up display and the at least one first device each include a corresponding one of the at least one second device comprising a transmitter and receiver to transmit and receive signals, and establishing the interaction further comprises:

performing the interaction via one or more from a group of a network connection and signals exchanged between the transmitter and receiver of the first heads-up display and the at least one first device.